AUC-maximized Deep Convolutional Neural Fields for Sequence Labeling

Learning from complex data with imbalanced label distribution is a challenging problem, especially when the data/label form structure, such as linearchain or tree-like. The widely-used training methods, such as maximum-likelihood and maximum labelwise accuracy, do not work well on imbalanced structured data. To model the complex relationship between the data and the structured label, we presents Deep Convolutional Neural Fields (DeepCNF), which is an integration of Deep Convolutional Neural Networks (DCNN) and Conditional Random Field (CRF). To handle the imbalanced structured data, we train DeepCNF by directly maximizing the empirical Area Under the ROC Curve (AUC), which is an unbiased measurement for imbalanced data. To fulfill this, we formulate AUC in a pairwise ranking framework and approximate it by a polynomial function and then apply a gradient-based procedure to optimize this approximation. We then test our AUC-maximized DeepCNF on three very different protein sequence labeling tasks the results confirm that maximumAUC greatly outperforms the other two training methods.